We evaluate the performance of four leading solutions for de-identification of unstructured medical text - Azure Health Data Services, AWS Comprehend Medical, OpenAI GPT-4o, and John Snow Labs - on a ground truth dataset of 48 clinical documents annotated by medical experts. The analysis, conducted at both entity-level and token-level, suggests that John Snow Labs' Medical Language Models solution achieves the highest accuracy, with a 96% F1-score in protected health information (PHI) detection, outperforming Azure (91%), AWS (83%), and GPT-4o (79%). John Snow Labs is not only the only solution which achieves regulatory-grade accuracy (surpassing that of human experts) but is also the most cost-effective solution: It is over 80% cheaper compared to Azure and GPT-4o, and is the only solution not priced by token. Its fixed-cost local deployment model avoids the escalating per-request fees of cloud-based services, making it a scalable and economical choice.

This blog post is co-authored by Vittorio Accomazzi, Chief Technical Officer (CTO) at IMS. This blog is part of a series in collaboration with our partners and customers leveraging the newly announced Azure Health Data Services. Azure Health Data Services, a platform as a service (PaaS) offering designed to support Protected Health Information (PHI) in the cloud, is a new way of working with unified data--providing care teams with a platform to support both transactional and analytical workloads from the same data store and enabling cloud computing to transform how we develop and deliver AI across the healthcare ecosystem. The first implementation of digital imaging techniques in clinical use started in the 1970s. Since then, the medical imaging industry has grown exponentially--over the last two and a half decades, there has been a significant development in image acquisition solutions, which has boosted image quality and adoption in different clinical applications.

Today, we take a giant step toward making the dream of interoperability in healthcare real. Microsoft is announcing the general availability of Azure Health Data Services, a platform as a service (PaaS) offering designed exclusively to support Protected Health Information (PHI) in the cloud. Azure Health Data Services is a new way of working with unified data--providing your team with a platform to support both transactional and analytical workloads from the same data store and enabling cloud computing to transform how we develop and deliver AI across the healthcare ecosystem. "Give me all the medications prescribed and connected home health device data with all the CT Scan documents and their associated radiology reports for any patient older than 45 with a diagnosis of osteosarcoma over the last 2 years." The above statement is a common request to health data managers.